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FOREWORD 



This report summarizes the concepts, approach, and developmental activity in 
establishing the first generation hardware/software subsystem (HSSS) of WorK Unit 
IMPACT, Prototypes of Computerized Training for Army Personnel. IMPACT is an 
advanced development project undertaken by the Human Resources Research 
Organization, designed to provide the Army with a system for computer-administered 
instruction (CAI). 

The National Science Foundation is also sponsoring HumRRO research on Instruc- 
tional Decision Models (IDMs), with additional support provided by the James McKeen 
Cattell Fund. This document is also intended to serve as the first report to the 
Foundation on the progress of the hardware/software subsystem toward implementing the 
initial IDM within the HumRRO CAI environment. 

The research is being conducted at HumRRO Division No. 1 (System Operations), 
Alexandria, Virginia, where Dr. J. Daniel Lyons is Director. Dr. Robert J e Seidel is the. 
Program Director. 

Technical contributions to the development of the HSSS were made by Mr. Jean A. 
Gameau, Mr. Leslie W. Willis, Mrs. Doris Shuford, Mr. John Stelzer, Mrs. Beverly Hunter, 
Dr. Felix Kopstein, Dr. Ronald Swallow, Miss Lala J. Curry, Mrs. Judith G. Compton, 
Dr. Harold G. Hunter, Mr. George R. Sedberry, and Dr. Robert J. Seidel. Mrs. Hunter 
and Mr. Martin L. Rubin contributed to the organization and writing of the report. 

The IMPACT project and the IDM research program follow on earlier HumRRO 
work in the same general area under Work Unit METHOD, Research for Programed 
Instruction in Military Training, and Exploratory Research 42, Organization of Instruc- 
tion. Principal publications under these research efforts include: Project IMPACT: 
Computer- Administered Instruction Concepts and Initial Development , by Robert J. 
Seidel and the IMPACT Staff, HumRRO Technical Report 69-3, March 1969; The 
Computer as Adaptive Instructional Decision Maker ; by Felix F. Kopstein and Robert J. 
Seidel, HumRRO Professional Paper 1-79, January 1970; Poject IMPACT: Description of 
Learning and Prescription for Instruction , by Robert J. Seidel, et al> HumRPtO Profes- 
sional Paper 22-69, June 1969; The Application of Theoretical Factors in Teaching 
Problem Solving by Programed Instruction , by Robert J. Seidel and Harold G. Hunter, 
HumRRO Technical Report 68-4, April 1968; Programed Learning: Prologue to Instruc- 
tion , by Robert J. Seidel, HumRRO Professional Paper 17-67, April 1967; and 
Computer-Administered Instruction Versus Traditionally Administered Instruction: 
Economics , by Felix F. Kopstein and Robert J. Seidel, HumRRO Professional Paper 
31-67, June 1967. 

Identification of products is for research documentation purposes only, and does not 
constitute an official endorsement by HumRRO, the Department of the Army, The 
National Science Foundation, or the James McKeen Cattell Fund. 

HumRRO research for the Department of the Army is conducted under Contract 
DAHC 19-70-C-0012. Computer-Administered Instruction research is conducted under 
Army Project 2Q063101D734. The IDM research being conducted under National Science 
Foundation sponsorship is funded under Grant GJ-774, Research on Instructional Deci- 
sion Models, with additional funds from the James McKeen Cattell Fund. 
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BACKGROUND 

In order to cope r/ith rapidly changing conditions and at the same time increase its 
posture of : iadiness, the U.S. Army may have to change some of its priorities, especially 
in the area of training its personnel, upon whom increasing demands will be made. With 
financial resources shrinking, less new equipment being developed, and a smaller, all- 
volunteer force in prospect, personnel will increasingly be recognized as the Army’s most 
vital resource, and training will assume a more dominant position. 

Among the problems faced by those responsible for training in the Army are broad 
student differences, decreasing numbers of skilled instructors, personnel fluctuations, and 
the need for flexibility and continual updating of courses. Computer-Adm listered 
Instruction (CAI) is expected to make a major contribution toward providing raining 
adequate to the task of dealing with these problems. 

The objective of Project IMPACT is to evolve a series of prototype systen. s of 
Computer- Administered Instruction in order to produce a prototype operational CAI 
system that is effective, efficient, and cost beneficial for use in Army training. A towil 
CAI system consists of four kinds of components: 

(1) The computer and its associated hardware devices 

(2) The computer software 

(3) Courses of instruction 

(4) Instructional decision models (IDM) 

The computer hardware and software, taken together, comprise a subsystem that is the 
subject of this report. 

Objectives 

The objectives for the IMPACT CAI hardware/software subsystem are, first, to 
provide, within the development time and cost constraints, a generalized and flexible tool 
for research and development of CAI instructional decision models and courses of 
instruction; second, and equally important, to provide computer hardware and software 
that is operationally efficient and economical for large-scale Army CAI use. 

The hardware /software subsystem is designed to provide the flexibility needed to 
develop courses in a variety of subjects of instruction, according to a wide range of 
instructional strategies, for a spectrum of Army student populations, within a framework 
of research development and experimentation. 

The hard ware /software prototypes must therefore provide a balanced capability that 
will satisfy the requirements for instructional development (needed to develop the IDM 
and course prototypes), and the requirements for efficient large-scale Army operations. 

Approach 

The approach taken toward the development of the hardware/software subsystem 
(HSSS) reflects a compromise between immediate and longer-range requirements. Off-the- 
shelf hardware and software components were used wherever possible, and tailored or 
modified to provide greater research flexibility and greater potential for operational 
efficiency and economy. 

The approach used to achieve the balance between development and operational 
requirements was to design the tools needed for research, and to implement these tools 
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and capabilities in computer hardware and software technology that can be expanded and 
refined to provide operational efficiency. 



Subsystem Description 

The hardware /software subsystem supports four main activities in a CAI environ- 
ment. These are: 

(1) Presenting instruction to students. 

(2) Implementing courses into CAI format 

(3) Evaluating students, courses, and instructional decision model. 

(4) Performing administrative functions in a school. 

In support of these activities, the subsystem includes the following components: 

(1) A general-purpose digital computer, usually classed as a medium-scale, 
third-generation computer (IBM 360 Model 40). This computer was selected because it 
has the general characteristics of, and is compatible with, larger-scale computers that 
would be required for operational CAI for the Army. 

(2) Twelve • athode ray tube (CRT) terminals for student-computer communi- 
cation. A commercially available CRT device (Sanders 720) was selected for its wide 
range of capabilities from a research standpoint and for its technological similarity to a 
family of terminal devices that are predicted to be the most economical for large-scale 
use in Army training, 

(3) Experimental devices for student-computer communication, including a 
computer-controlled film projector with addressable frames; a tablet for handprinted 
input to the computer; and a voice frequency analyzer which enables a student to speak 
to the computer in single words. 

(4) Direct access storage (IBM 2314 disk) for storing course material, student 
performance records, and IDM. These storage devices make it possible to have several 
thousand hours of instruction available online at a time, a necessity in an operational 
Army school environment. 

(5) IMPACT-Coursewriter, a computer software package that controls the 
operation of the hardware to perform CAI functions. A CAI software package already 
developed by IBM (Coursewriter III) was modified to provide greater capability for 
collecting data, and to interface with remote CRT terminals that would be needed in an 
operational Army setting. 

(6) Zeus, a computer software monitor that makes it possible to operate 
multiple CRT terminals from locations remote from the central computer. 

(7) EDITOR (Entry on Disk of Instructional Text for Online Retrieval), a set 
of computer commands that enable authors to create, modify, and retrieve instructional 
materials on the CRT. 

(8) DIRECTOR, a software supervisor that retrieves course content from disk 
and transmits it to individual students at the appropriate time. DIRECTOR is the 
interface between course programs and IDM (which decide what instruction a student 
needs next) and the student. 

(9) IMPACT Data Evaluation System (IDES), a software package that manages 
the data collected, stored, and processed for evaluation of students, courses, and IDM. 

(10) File Activity Control System (FACS), a software package that assists 
authors in preparing content for CAI courses. 
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Project IMPACT— 
Computer- Administered Instruction: 
Description of 

the Hardware/Software Subsystem 



Part 1 



INTRODUCTION 



THE ARMY AND COMPUTER-ADMINISTERED INSTRUCTION 

In order to cope with rapidly changing conditions within its organization and at the 
same time increase its posture of readiness, the TJ.S. Army of tomorrow will have to 
change some of today’s priorities. Greater emphasis will especially have to be placed on 
the training of its personnel, upon whom increasing demands will be made. 

With shrinking financial resources, less equipment development, and a smaller, 
all-volunteer force in prospect, the Army will increasingly recognize personnel as its most 
vital resource, and training will assume a more dominant position than it does today. 

Army training will have to be adequate to the task of dealing with widespread 
differences among students, especially in the event of a mobilization buildup and the 
resulting mass training requirements. To those responsible for Army training, individu- 
alized training is the key to meeting these requirements. With fewer skilled instructors 
available, the fluctuation in personnel, and the requirements for flexibility and continual 
updating of formal and on-the-job courses, Computer- Administered Instruction (CAI) with 
its unique capabilities is expected to assume a major role in providing the training 
required for the Army’s readiness goals. 

The use of general purpose digital computers for instruction is a relatively recent 
development in educational technology. “Computer assisted Instruction” as it is generally 
called holds vast promise for providing a more effective, efficient, and economical 
instructional system than traditional instructional methods and systems. The promise lies 
in the potential of the computer for adapting instruction to the momentary needs and 
capabilities of the individual student, for use at his own pace. 

The U.S. Continental Army Command (CONARC), which is responsible for training 
in the Army’s school system, has established a long-range plan for automatic data 
processing (ADP) that includes the use of the computer’s capability for training. The plan 
contains CON ARC’S philosophy concerning CAI: 

The philosophy of CONARC concerning Computer Assisted Instruction remains essen- 
tially the same as the philosophy presented in the briefing for the Assistant Secretary of 
Defense for Education in December 1967. It stated that management controls at all levels of 
command must insure that the integration of CAI into the Army school system is free of 
duplication of effort, is economical in the use of resources, and is progressively leading to 
the accomplishments of the Army objective which is to provide computer operational 
capability for the conduct of instruction for all Army Schools. To guarantee strict adherence 
to this philosophy, USCONARC exercises a progressive developmental policy which 
encourages projects for investigations in discrete applications of the computer in support of a 
training function to insure that the established feasibility of this technology is strengthened 
and refined. It has been ascertained that further investigation should be undertaken in several 
CAI modes, notably drill and practice, simulation, and student dialogue. On this basis, initial 
projects for each of these modes are being conducted at separate USCONARC schools. 
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Additionally, all school faculties must be trained for identification and development of CAI 
applications to assure attainment of the objectives listed above. 1 

To support these objectives, Project IMPACT (Prototypes of Computerized Training 
for Army Personnel) has been established as an integrated, multidisciplinary CAI effort. 
The objective of the effort is to evolve, through cyclical development and evaluation, an 
effective, efficient, and economical Computer-Administered Instructional system. The 
term “ Computer-Administered ” is used because the computer in this system houses the 
controlling and adaptive instructional decision model (IDM). 

Problem areas addressed include computer system capabilities, CAI language needs 
and CAI potential, and, of prime importance, the meaning of instructional strategies and 
their relationships to learning processes. Project IMPACT is unique in that it considers the 
total instructional system. The problem for any instructional agent (human or machine) is 
to take optimal action in line with an overall “best” strategy for transmitting to the 
student information uniquely relevant to him. If proficiency criteria are to be attained 
effectively and efficiently, recurrent decisions concerning these instructional actions must 
be made relative to (a) the subject matter being taught, (b) the specific student, (c) the 
momentary circumstances, and (d) the available options (communication channels). 

The potential of CAI will be realized only with a systematic, persistent, and iterative 
scientific effort leading to a synthesis of principles into a potent model of the instruc- 
tional decision process. With this realization the new system of instruction can be molded 
into a cost/effective undertaking. 

This total instructional system is considered in Project IMPACT over four phased 
development cycles. The heart of this total system CAI effort is the iterative development 
and testing of instructional decision models (IDMs). During the four cycles of develop- 
ment and testing, the comp aents of the effort (hardware, software, subject matter, and 
IDM) are to be revised and updated. The first two cycles comprise the development and 
the evaluation of the “breadboard” (preliminary) CAI system. To be undertaken in cycles 
three and four are the synthesis and implementation of the refined components into a 
prototype operational CAI system^what IMPACT will ultimately provide to the Army. 

This advanced development effort will supply the Army with its own capability for 
developing sound, effective CAI materials. These generalized logic subprograms will be 
designed and documented for use by technically unsophisticated personnel such as 
instructors, lesson designers, and subject-matter experts so that they will be able to 
modify the course for their particular purposes. Through the development of a useful 
family of instructional decision models the programs of instruction will be adaptable to 
the moment-by-moment capabilities of the individual trainee and the content made 
relevant to his specific job requirements. 

The products will also include design requirements for hardware configuration for 
operationally implementable Army CAI. In the software area, computer language will be 
developed to facilitate interaction between author and computer, trainee and computer, 
and administrator and computer. User documentation will be provided to simplify the 
implementation of these products. 

Since the approach of Project IMPACT is a total systems view of CAI, all 
components of the system, including hardware, computer software, instructional content, 
instructional strategies, and learning principles are included in the development of the 
prototype and in the research studies conducted. Project IMPACT, which began in FY 
1968, is described in detail in Project IMPACT: Computer Administered Instruction 



headquarters, U.S. Continental Army Command, Director of Management Information Systems. 
"USCONARC Long Range ADPS Master Plan (Schools)”, 27 December 1968. Part III, “Narrative”. 
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Concepts and Initial Development (1). In this report, the present IMPACT hardware/ 
software subsystem is referred to as IMPACT-A HSSS. IMPACT-A HSSS is a “first- 
generation” prototype, not intended for implementation in Army schools. On the basis of 
experience with IMPACT-A, specifications are being prepared for an operational CAI 
HSSS. This next generation is referred to here as IMPACT-B HSSS. When the term 
“system” is used, it refers to the total CAI system including HSSS plus IDM, course of 
instruction, instructional strategies, and students. A Glossary of terms used in this report 
is provided in Appendix A. 



PURPOSE OF THIS DOCUMENT 

This document provides an introduction to the capabilities of the IMPACT-A HSSS. 
It does not describe specific courses of instruction, IDM, research programs, or instruc- 
tional strategies that are a part of the total system development at Project IMPACT. 

This report is the first in a series designed to provide introductory information to 
Army training experts, prospective managers of CAI installations, and instructional system 
planners. It should help to answer such questions as: 

(1) What tools does the HSSS provide for administering instruction? 

(2) What instructional environment is provided to the student? 

(3) What tools does the HSSS provide for instructional programmers? 

(4) What capabilities are provided for evaluating the CAI system and the 
courses? 

(5) What are the major constraints in the system? (For example, dependence 
on specific hardware or subject matter.) 

(6) What hardware and software resources are required to implement a given 
component? 

Other reports, now in preparation, will describe the hardware and software sub- 
system in more detail, including course implementation capabilities, facilities for course 
administration, and evaluation. 



SALIENT FEATURES OF THE HARDWARE/SOFTWARE SUBSYSTEM 

The HSSS provides a generalized capability for administering, developing, and eval- 
uating individualized courses of instruction and tests. In the IMPACT CAI system, the 
computer acts as an instructor or tutor in that it interacts with students individually; 
presents material to the student as needed, and evaluates the student’s progress in 
achieving course objectives. It is thus a “tutorial” CAI system, as distinguished from 
other uses of computers in training, such as in simulations or games, or as a problem- 
solving tool. 



OPERATING MODES 

The HSSS is designed to operate in four modes, corresponding to different activities 
performed in a CAI center, and illustrated by the four segments of Figure 1. The HSSS 
operates in these modes either separately or concurrently. 

In Figure la the components that operate in Student Mode are shown. This is the 
mode in which instruction is being administered to students. 
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Overview of IMPACT CAI System Components and Operating Modes 
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The system components in operation in Author Mode are shown in Figure lb. In 
this mode, the instructional content is prepared by the authors who then ha\5 it stored 
in the text files. 

The components utilized in the Evaluation Mode , in which educational researchers 
evaluate instruction and IDM, are illustrated in Figure 1c. 

In Figure Id the components used in the Support Mode are shown. In this operating 
mode, student performance records are queried by school personnel for administrative 
information. 



PRIMARY COMPONENTS 
Hardware 

(1) The central computer, an IBM 360 Model 40 (moderate-size, third-generation 
computer). 2 

(2) Student stations for input and output to the computer, including cathode ray 
tube (CRT) and keyboard, a film projector, hard-copy devices, and experimental 
hand-printed and voice-input devices. 

(3) Direct access storage (IBM 2314 disk) for course material, student performance 
records, and instructional logic. 

(4) Equipment required to support communications between computer and remote 
student stations. 



Software 

(1) Zeus, a computer software supervisor that supports CRT terminals at locations 
remote from the central computer. 

(2) EDITOR (Entry on Disk of Instructional Text for Online Retrieval), a simple to 
use yet powerful set of commands that enable course authors to create and 
modify course materials on the CRT and have them stored on disk (text files) at 
the central computer. 

(3) DIRECTOR, a software supervisor that retrieves the appropriate course content 
for an individual student and transmits it to his student station. DIRECTOR is 
the interface between course programs and IDM (which determines what instruc- 
tion to present next), the course materials, and the student. 

(4) IMP ACT-Course writer, a generalized CAI software package that includes: 

(a) IBM Coursewriter III author language. 

(b) Coursewriter III processor augmented with functions required for IMPACT 
research. 

(c) An input-output control program expanded by IMPACT to support remote 
CRT terminals. 

(5) IMPACT Data Evaluation System (IDES), a data management package used to 
structure and retrieve student and course performance data. 

(6) File Activity Control System (FACS), a software package that assists authors in 
preparing the content of CAI courses. 



identification of products is for research documentation purposes only and does not constitute an 
official endorsement by HumRRO, the Department of the Army, the National Science Foundation, or 
the James McKeen Catted Fund. 
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Part 2 



IMPACT-A HSSS CAPABILITIES 
FOR ADMINISTERING INSTRUCTION 
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When the computer is administering instruction to students it is said to be in 
“student mode” (Figure la). This section of the report describes student mode operation 
and capabilities. 



OPERATION OF THE SYSTEM IN STUDENT MODE 
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The major components of the system that operate in student mode are shown in 
Figure 2. Circled numbers in the discussion that follows correspond to the numbers on 
the figure. 

HSSS Components Involved in Administering Instruction 
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The student keys his input to the computer (e.g., an answer to a question or a 
request for assistance) through either a CRT keyboard or the 1050 typewriter keyboard 

( 1 ) . (This discussion will refer to the use of the CRT only.) 

The student input is received in the computer by Zeus (2) . Zeus signals the 
IMPACT-Coursewriter (3) processor that the message has been received and is ready to be 
processed. 

IMPACT-Coursewriter determines which student has transmitted this response, and 
what stage he has reached in the course (from the records at (4)). IMPACT-Coursewriter 
processor then obtains the appropriate segment of course logic instructions from the 
course program (5) . IMPACT-Coursewriter executes instructions in the course program 
which tell how to interpret the student’s input (S) , thereby providing the student 
response portion for the IDM decision on what action is to be taken next (7) . The IDM 
decision is transmitted to the course program, which selects the next course program 
instruction to be executed (§) . 

IMPACT-Coursewriter output control transmits this instruction to DIRECTOR (9) . 
DIRECTOR finds and retrieves the appropriate material from the text files . 
DIRECTOR transmits that display, through Zeus and the IBM 360 Operating System, to 
the student’s CRT (ll) . The frames of a film to be presented along with the CRT 
display are computer controlled. A special electronic device (not shown in Figure 2) 
interprets the computer commands and activates the film projector (Perceptoscope) @ 

(2) . The student reads the displays and makes the response called for by the instruction 
QS . As shown in Figure 2, many students may be instructed simultaneously, from one or 
many courses. 



CAPABILITIES FOR INSTRUCTIONAL OUTPUT 

The HSSS output capabilities required for a given course vary depending on the 
subject matter being taught, the skills of the students, and the instructional strategies 
being used. An objective of Project IMPACT is to provide a range of output capabilities 
in the HSSS, in order to allow for teaching a variety of subject matter and to provide a 
broad base for CAI research. 

The output capabilities are a combined function of hardware devices and the 
computer software. The three basic output devices presently used are a CRT display, 
teletypewriters, and a film/slide projector. The major portion of the software is in Zeus 
and DIRECTOR. 



HARDWARE DEVICES 



CRT Display 

The CRT used for IMPACT-A HSSS is the Sanders 720, which has a wide range of 
capabilities compared to other commercially available CRTs. These capabilities are being 
evaluated as they are used, so that the IMPACT-B CRT can be specifically tailored to 
CAI requirements. The Sanders 720 can be located at distances remote from the central 
computer via telecommunication lines (given the necessary software). It can project up to 
1,024 alphabetic, numeric, and special characters on a page. These characters may be 
distributed over 2,080 screen positions. 

The primary asset of this CRT for instruction is its large repertoire of formatting 
capabilities. The display screen can be divided into variable length logical “blocks” 
(Figure 3). These blocks can be used for various instructional output purposes, such as 
explanation, question, response, feedback, or remediation. A CRT display is illustrated in 
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Schematic of CRT Block Structure (Example) 

f 
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I Text of question. 


El 




j Student response block. I 


LVvXl 



Stondard reminder of use 
of the CRT. 

HINT transmitted when 
student does not respond 
within time limit. 



Figure 3 



Figure 4, and Figure 5 shows the same text display, with a feedback message that has 
been added to it by DIRECTOR. This is accomplished without retransmitting the 
information from the prior display. (Greater detail on the Sanders 720 is found in Part 6 
of this report.) 

Hard-Copy Devices 

Two types of hard-copy devices, with different purposes, may be used fcj output to 
studer ts. One is a teletype device used to copy the display or “page” that is on the CRT 
screen. This device is under the control of the CRT keyboard and may not receive 
messages directly from the computer. This capability may be used by a student to get a 
copy of a homework assignment or some other material for use “offline”. 

The other hard-copy device is the IBM 1050 typewrtier terminal. 3 Output on this 
device is under computer control. The typewriter receives text from the computer and 
the student responds through the 1050 keyboard. Formatting material for the IBM 1050 
is substantially different than with the CRT. Material prepared for presentation on the 
1050 can be used, essentially unchanged, on the CRT. However, material prepared for 
CRT cannot be presented on the 1050. 



3 The 1050 terminal configuration was deleted from the system, for budgetary reasons, as of July 
1970. However, their use in the system is described. 
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CRT Display With Question for Student 
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CRT Display With Student Response and 
Feedback After Student Response 
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Film/Slide Projector 

A motion sequence film or a frame of film may be projected in conjunction with a 
CRT display. The projector (Perceptoscope) operates under commands that are trans- 
mitted through the CRT circuitry (3). 4 Hence, the addressable frames and motion 
sequences may be synchronized with the instruction appearing on the CRT. The Per- 
ceptoscope provides the capability to project graphic and schematic information that is 
not easily conveyed through the alphanumeric text of the CFT display, and it may also 
be used m situations when color is a key element to communicating the instruction, to 
show a short movie, or to step through a sequence of frames at a controllable speed. Use 
of this device in a course being run under Project IMPACT is illustrated in Figure 6. 



Use of the Perceptoscope 




SOFTWARE SUPPORT FOR THE CRT 

Zeus and DIRECTOR perform many functions needed to support multiple CRT 
terminals at remote locations. One of these functions is to retrieve course material from 
direct access storage and project it on the CRT screen to the student. DIRECTOR 
receives commands from the course program as to what instructional material is to be 
transmitted next. DIRECTOR locates that material in a file of preformatted text and 
transmits it to the student’s CRT. These displays (elements of instructional text) are in a 
file separate from course logic. They may be recalled many times within a course of 
instruction. The software can string together various text elements to form a variety of 
total displays. This is one of the mechanisms by which a specific instructional presenta- 
tion may be tailored to the needs of a particular student at a given point in time. 

Zeus and DIRECTOR are designed to accept commands from course programs that 
contain the logic, or strategy, of a course of instruction. At present in Project IMPACT, 
course programs are prepared using the IMPACT-Coursewriter language. However, Zeus 
and DIRECTOR are designed to accept commands from any language; they are not 
restricted to IMPACT-Coursewriter operations. Any programming language that operates 
on the IBM 360 and is suitable for expressing the logic of a course of instruction may 
operate in conjunction with Zeus and DIRECTOR. 

4 The Perceptoscope cannot be used in conjunction with the IBM 1050 typewriter. 





TYPES OF INSTRUCTIONAL OUTPUT 



The foregoing devices and computer software provide for all types of instruction to 
be presented to the student, including: 

(1) Explanatory text and graphics 

(2) Remedial text and graphics 

(3) True-false, multiple choice, and constructed response questions 

(4) Quizzes and tests 

(5) Study or homework assignments 

(6) Exercises and drills 

(7) Feedback on student performance; attention-getting cues 

(8) Glossary definitions 



CAPABILITIES FOR STUDENT INPUT 

This section describes the means by which the student communicates with the 
computer, the types of input the student may enter, and the interpretation of student 
input by the computer. 



HARDWARE DEVICES 

The primary device for student input is the keyboard associated with the Sanders 
720 CRT. (If the course is being administered via a 1050 typewriter, the input medium 
is, of course, the 1050 keyboard.) 

The primary difference between typing on a CRT keyboard and typing on a 
typewriter is that with the former the cursor must be properly positioned on the CRT 
screen before the student can respond. The cursor is a pointer to the screen position 
where the input is to be entered. It appears as a single character, blinking underline. The 
IMPACT CAI system has three features that minimize the problem of cursor positioning 
for the student: 

(1) DIRECTOR pre-positions the cursor at the exact spot on the screen where 
the student is to begin his response. 

(2) The terminal is equipped with an electronic pen which the student may use 
to move the cursor to any position on the screen. (For example, the student may wish to 
move the cursor around to answer a series of questions in a different order than they are 
presented on the screen). 

(3) The display may be formatted in a way that causes the cursor to auto- 
matically skip from one answer space to another. Figure 7 is an example of an 
instructional display that takes advantage of this capability. 

The other devices-^the tablet and voice input—are being developed to further open 
the channels of communication from the student to the computer. The tablet, called the 
“Sylvania Tablet” enables a student to hand print, rather than type, an input. Experi- 
mentation is underway to determine the extent to which this device improves communi- 
cation between the student and the IDM. The voice input device allows a student to 
speak words from a prestored list, instead of using some form of hand input. The present 
vocabulary is limited to 25 words because of memory limitations. The recognition error 
rate is 1 %. 



Use of CRT Formatting to Control Cursor Positioning 
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THE PRIMARY OUTPUT CAPABILITY OF THE SYSTEM. THE 




PROVIDES AUXILIARY OUTPUT. THE 
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tThe cursor automatically moves to point b after blanks at a. 
ire filled in. Similarly with c and 6 . Students may override 
automatic cursor movement by using the electronic pen.) 
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Text of Question 



Student response block 



Figure 7 



TYPES OF STUDENT INPUT 

The types of inputs the student may make are summarized in Figure 8. The uses of 
various types of input are in every case at the option of course designers, with the 
instructional strategy being the determining force. 

The glossary requests may be made before the student answers a question, while he 
is reading the instructional text, or, at the course designer’s option, during tests. The 
glossary request capability is built into DIRECTOR. It represents a first step in the 
direction of a generalized information retrieval capability for students. 

If a student is having difficulty with a segment of the course, or finds any aspect of 
the system frustrating or confusing, he may choose to key in a comment These 
comments are later reviewed by appropriate operations personnel or course authors, and 
used to make improvements in the system or to determine need for individual student 
counseling. (See Part 4, Evaluation.) 
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Summary of Types of Student Input With IMPACT-A HSSS 



Class of Input 


Specific Types 


Constraints 


Answers to Questions 


Multiple choice 

True/false 

Matching 

Constructed response 
single word or 
several words 
Graphics formed from 
Sar tiers 720 characters 


Lin© drawings 
may not be 
input, a 
constraint of 
the Sanders 720. 


Inquiries 


Glossary requests 

Requests for specific 
topics of instruction 
for review, or assignments. 

“HELP” requests, for 
a hint or explanation 


Definitions 
are limited to 
252 characters. 


Comments 


Any remark student 
wants to make about 
the course or system, 
may be entered at any 
time. System will not 
confuse comments with 
other responses. 


Student receives 
nc immediate 
feedback. The 
comments are 
reviewed offline 
by instructors. 



Figure 8 



INTERPRETING STUDENT INPUT 

The flexibility of the tools available / interpreting student responses has a tre- 
mendous impact on the responsiveness of the CAI system to the student’s learning 
requirements. At the simplest level, the expected responses to a question are prestored in 
the course program. The student’s response is compared to the prestored words, and if \ 
precise match is obtained, the student receives a correct answer score. 

In order to eliminate some of the rigidity inherent in character-by-character com- 
parisons, IMPACT- Course writer provides tools for scanning a response for key words or 
character strings, which effectively allow for variations in spelling and punctuation. 
However, the problem of having to prestore each word or phrase at each response point 
in the course still remains. 

Project IMPACT has added a function called “SEEK” to Coursewriter language, 
which makes it easier to specify alternative responses. SEEK enables a course author to 
specify a list of response words that are common to a sequence of instruction and are 
used repeatedly . 5 A course segment coded with fn SEEK had six times fewer computer 
instructions than the same segment without fn SEEK. 

5 Function SEEK also has other uses, described in IMPACT’S Coursewriter function documentation. 
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Categorization of Responses 

Once the system has been able to identify a student response as meaningful, or 
anticipated, it can then categorize and store it. The IMPACT-A HSSS provides 80 
counters for interpretive information for each student (an extension beyond Coursewriter 
Ill’s E’O counters). 

The use of these counters for response interpretation by the IDM is a matter of 
establishing conventions for a particular instruction segment. For example, it may be 
determined that there are 20 basic concepts in a particular segment. Separate counters 
may be established, by convention, to represent those concept categories. Each student 
response would be interpreted as indicating whether or not a particular concept was being 
grasped by the student. The counters would then be used as a factor in making decisions 
as to what instruction or test question to present next to the student. 

"State of Knowledge" (SOK) 

Other mechanisms may be built into courses for interpreting student responses. For 
example, in Project IMPACT’S course in COBOL, each student’s response includes a value 
for the confidence the student expresses in his answer. The response interpretation 
programs assign a “state of knowledge” (SOK) value to the response, which is a function 
of the correctness of the response and the student’s confidence in the answer. The SOK 
is used by the IDM in making decisions about what instructional action to take next. 



CAPABILITIES FOR MAKING INSTRUCTIONAL DECISIONS 

The ability to make decisions that result in instruction tailored to the individual 
student is at the heart of Project IMPACT. This capability, the Instructional Decision 
Model (IDM), consists of rules for the computer system to use in deciding what to do 
next (4). 

The major elements of the decision-making process are shown in Figure 9. The 
decision factors for use by the IDM, shown in Figure 10, encompass student performance 
data during the course of instruction as well as a profile of the student obtained at 
registration time. The theoretical underpinnings of an IDM are discussed in & number of 
works (5, 6, 7). 

Analysis and decision-making actions taken by the IDM during the time the course is 
being administered to the student take place online . Additional analysis and decision 
making are performed offline , as a process separate from course administration, either by 
the computer (in batch operation) or manually. In other words, some decisions are made 
in real time, that is, in time to affect the instructional process for the student. Other 
decisions are made out of the time frame of actual course administration. 

The decision factors listed in Figure 10 are those the IMPACT-A HSSS provides 
capability for using, either online or offline , as indicated. The offline factors are being 
studied as a pari; of Project IMPACT, and depending on the outcome of this research, the 
IMPACT-B HSSS will be expanded to provide for online use of the significant decision 
factors. For example, if response latency is demonstrated to be an important factor in 
deciding whether to present remedial instruction, the HSSS will be expanded to allow for 
latency as an online decision factor. 

Entry characteristics, such as previous training and scores on standard tests, may be 
used to determine difficulty level and other characteristics of the material to be 
presented. Under IMPACT-A, researchers provide the interpretation, prior to registering a 
student for the course; however, under IMPACT-B much of this would become an online 
function. 
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Major Elements in Decision Making 



DECISION FACTORS 

Previous Response(s) 
Interpretation 
Latencies 

Student* s confidence 
Number of attempts 

Response Patterns 
Quiz Test scores 
State of knowledge 

Entry Characteristics 
Biographical data 
Educational level 
Cognitive functions 
Experimental group 
Aptitude tests 

Army Classification Battery 




DECISION RULES 

On the basis of 
decision factor 
values for this 
student, select 
appropriate 
instructional 
option, according 
to course 
strategy. 




INSTRUCTIONAL OPTIONS 

Next topic 
Remedial instruction 
Review 

Drill, exercises 
Test, quiz 

Homework assignment 
Feedback on performance 
Hints, suggestions, cues 
Answer again 
Glossary definition 
k Sign off 
Call instructor 



SELECTED OPTION 



Student Requests 
INFO 
Help 
Review 
Topic 



Figure 9 



Every time a student makes an input, a decision point is reached. The greater the 
variety and number of instructional options that are available, the more refined the 
instruction can be in terms of meeting individual student needs. The kinds of instruc- 
tional options that may be provided by IMPACT at any point in the course are 
theoretically unlimited. From the point of view of the HSSS, review material, drills or 
exercises, tests, new topics, homework assignments, and so on, are all the same. The 
capabilities of DIRECTOR for retrieving display elements from text files enable the 
IMPACT -A HSSS to provide a very high degree of flexibility in the number and kinds of 
options available at any decision point. 

The decision-making process by which an instructional option is chosen and specific 
actions taken is illustrated in Figure 11. The output of the response interpretation 
consists of decision factor values stored in a series of counters. The counters are then an 
input to the IDM which relies on prescribed rules to make a choice among options. The 
decision is communicated to the Course Program, which in turn takes action through 
DIRECTOR. Then the student receives the tailored instruction presentation. 
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Decision Factors in IMPACT-A HSSS 



Class 


Factor 


Description 


Analysis 


Decision 

Making 


Previous 


Anticipated 


Is response recognizable? 


Online 


Online 


Response (s) 


Interprcto- 

tion 


Is a particular error or 
insight implied in the 
response? 


Online 


Online 




Latency 1 


Time from transmission 
of question to student's 
acknowledgement. 


Online 


Offline 




Latency 2 


Time from student acknow- 
ledgement of question to 
receipt of his response. 


Online 


Offline 




Confidence 

Number of 
attempts 


Student's expressed con- 
fidence is his answer 


Online 

Online 


Online 

Online 


Response 

Patterns 


Quiz, test 
Scores 


Individual scores or 
changes from test to test 


Online 


Online 




State of 
knowledge 


Increasing, decreasing 
state of knowledge 


Online 


Online 


Entry 


Biographical 




Offline 


Offline 


Characteris- 

tics 


Educational 


Grade level, previous 
training 


Offline 


Online 




Cognitive 

Functions 


Results from psychometric 
tests 


Offline 


Offline 




Experimental 

group 


Identity of student in an 
experimental or control 
group in a special study. 


Offline 


Online 




Aptitude tests 


Results from aptitude tests 


Offline 


Offline 




Army Class- 
ification 
Battery 


Results from Arnr.y Classification 
Battery 


Offline 


Offline 


Student 


INFO 


Request definition of term 


Online 


Online 


Requests 


Help 


Request help with a question 


Online 


Online 




Review 


Request review of a topic 


Online 


Online 




Topic 


Request topic of instruction 


Online 


Online 



Figure 10 
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Steps in Decision Making in the IMPACT-A HSSS 







Figure 1 1 



TESTING 



The same capabilities that are used to administer instruction to students may also be 
used to administer tests. Quizzes, as well as phase or other criterion tests, may be 
automatically administered to the student at the time appropriate for him. 



ADMINISTERING TESTS 

Tests may be administered in a variety of ways. Test questions may be presented to 
the student on the CRT screen in much the same fashion as a paper-and-pencil test. 
Standard paper tests may be given to the student by his instructor, and the student may 
enter his answers on the CRT (or 1050 typewriter) keyboard. Where a performance test 
requires the student to manipulate offline devices (e.g., electronic test equipment), the 
student may enter answers or result values on the CRT keyboard. Multiple choice, 
true-false, or single or multiple word constructed response tests may be used. Multiple 
forms or versions of a test may be stored in the HSSS and different forms administered 
to different students, on either a random or preplanned basis. 



SCORING AND GRADING 

Students’ answers to test questions are scored by the computer as soon as they are 
entered. In a course programmed in Project IMPACT, the scoring takes into account not 
only the actual answer given to a question, but also the confidence the student expresses 
in his answer (1, p. 18). 

Scores may be computed for single answers, groups of answers, or any combination 
of answers. The grades on the test may be computed immediately by the computer 
system, or the raw scores may be provided to the instructors for their analysis and 
grading. The student may be informed of his score immediately after each question, after 
a test segment, after a test, or not at all, depending on school procedures and require- 
ments. 



STORING AND RETRIEVING TEST SCORES 

The computer stores the test scores and/or grades in the student’s record. Through 
the IMPACT Data Evaluation System (IDES), discussed in Part 4 of this report, test 
scores and grades may be retrieved in a variety of forms for different purposes. An 
individual’s total test score or scores on individual test items may be printed out, or a 
listing of all students’ scores and/or answers to individual test items may be retrieved. 
Other data may also be retrieved, such as the amount of time it took a student to answer 
a question, and, if relevant, the number of times he attempted to answer it. 



Part 3 



IMPACT-A HSSS CAPABILITIES 
FOR IMPLEMENTING COURSES 



This section of the report describes the HSSS capabilities that assist in preparing 
CAI courses of instruction. 
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OVERVIEW 

The major development steps in preparing a CAI course are shown in Figure 12, in 
gross terms. The term “instructional programming” is often used to describe the last two 
steps shown in the figure, writing and implementing CAI segments. Both steps are 
someCmes performed by the same person or persons. However, this presentation is 
directly concerned only with course implementation, the last step shown, which is in 
direct interface with the HSSS. The efficiency and cost of performing this step is directly 
related to the capabilities of the HSSS. 

In Figure 13, the three major steps involved in course implementation are shown— 
formatting and entering CRT displays, coding and input of the course program, and 
testing and debugging. By separating these three steps, the IMPACT HSSS provides for 
more efficient and economical course implementation relative to other CAI systems. 



Overview of CAI Course Development Steps 




• Strategies 



characteristics 



“Instructional Programming” 



Figure 12 
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Major Steps in Course I n>plementation 



DISPLAYS PROGRAM TEST 




Figure 13 
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INSTRUCTIONAL DISPLAYS 



The main HSSS components used to format and store CRT displays in the computer 
are shown in Figure 14. The person who is creating or modifying instructional content 
sits down at the CRT terminal and types in the exact text and format characters desired. 



IMPACT-A HSSS Components Involved in Formatting and 
Storing CRT Displays 




On I ine creation, 
retrieval, revision 
of instructional 
content and format. 



EDITOR COMMANDS (AUTHOR MODE OPERATIONS) 

Through a simple command language (EDITOR), 6 the person formatting the page 
tells the computer to store it on file on magnetic disk (the “Author Mode” operation 
indicated in Figure lb). EDITOR is a generalized display handling capability, which 
operates independently of the CAI system. Hence it may be used for a variety of 
applications other than CAI. Sample EDITOR commands are shown in Figure 15. 



The use of EDITOR is described in a HumRRO report, in preparation, on preparing and managing 
instructional content in the IMPACT text-handling subsystem. 
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Sample EDITOR Commands 
Command 

CREATE D9999/MUSICSCALES 1 
COPY D8888,D8889, ‘SHARPS' 

MODIFY D9999 

DISPLAY D9999 
DELETE D9999 
ROLLON F7777,4, 1 



Explanation 

Create a display called “musicscales” 

Make a copy of o display already stored, 
and give it the new name “sharps” 

Bring display D9999 to the screen; 
replace it with the modifications made. 

Bring display D9999 to the screen. 

Delete D9999 from the file. 

Retrieve display F7777 from the file; 
“roll it on" to the CRT screen in 
Block 4. Position the cursor at 
Block 1. 



Figure 15 



Example of a Display Printout Generated by FACS 



OATE PRINTED 07/20/70 FACS 

OISPLAY NUMBFP TITLE 

001050 01050 



OPERATION OATE TIME AUTHOR 

MOOIFIEO 07/20/70 09/50/10 PICK 



PAGE 



l 



LINE 

1 

2 

3 

4 

5 

6 
7 

a 

9 

10 

11 

12 

13 

14 

15 

16 

17 

18 

19 

20 
21 
2? 

23 

24 

25 

26 
27 
2Q 

29 

30 

31 

32 



NUMBER OF BLOCKS* 6 
0E LAY LINE: 

1 2 3 * -4 — 5 6 7 R 9 0 1 2 3-- 

60ICK 095010640070201 1 01050 ¥¥£CCby9bbbbbbMybax*«*MMMMM 

#*#aooobooo^^^LbUHtitiuiibbbUddx«**«i««i*i**<MiAitioooanooai^c£.£,umiiidddadutitiaK*«««««««««««M««t'po?aocya^^c.bbaddddddddddX(i(i«*«««A««4»* 

• ••UOOObOOOil^y^mCltlNFO- MLFT'S REVIEW SO«F OF ThF 10FAS 1 HAVE PRFSFNTEO.££.ttCO M °l-*TF THE 

SENTENCFS BELOW WITH THE FOLLOWING WORDS : £££bdC0B0LUC0MPUTERb04TA P«OCESSINGbPRnGRAM£££bGI VE A SHOW OF CQNFIOENCE AFTER EACH ANSWER. 

PRESS SEND AFTER EACH ANSWER .EbUbbbyObbbiftlbbbb ££1. HANDLING LARGE AMOUNTS OF INFORMATION IS CALLFO££2. A MACHlNF THAT CAN 

PROCESS OATA RAPIDLY IS A££3. A SERIES OF STEPS FOP OlRECTlNG COMPUTERS IS A££4, THF PROGRAMS YOU WRITE WILL BE IN THE LANGUAGE £ 
££PRE5S SENO.a^mCC C 

£ 

t It 



! 2 3 4 5 6 7 0 

LET'S PEV1FW SOME OF THE IDEAS I HAVE PRESENTED. 

COMPLETF THE SENTENCES BELOW WITH THE FOLLOWING WORDS:. 

COBOL COMPUTER OATA PROC SSING PROGRAM 
GIVE A SHOW OF CONFIOEVCE AFTFR EACH ANSWER. PRESS SENO AFTER EACH ANSWFR. 



1. HANOLING LARGE AMOUNTS OF INFORMATION IS CALLEO XIIMMIMMIIMI 000 000 

2. A MACHINE THAT CAN PROCESS OATA RAPlOLY IS A XCM*MMi*MiiM 000 OCO 

3. A SERIES OF STEPS FOR DIRECTING COMPUTERS IS A XlflIMfllMIIIII 000 OCO 

4. THE PROGRAMS YOU WRITE WILL BE IN THE LANGUAGE XMiiiMiiiiMiM OOO 000 

PRFSS SENO. 



INFO- 

1 2 3 4 5 6 7 B- 



1 2 3 4 S 6 7 8 9 O 1 2 3- 



001050 



TITLE: D1050 



Figure 16 
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FILE ACTIVITY CONTROL SYSTEM 
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Thousands of display pages are usually needed in a long course of instruction. FACS 
(File Activity Control System) is a set of computer programs that provides information 
about the display pages that have been stored on disk. FACS helps in editing and 
coordinating the displays. The following are examples of the information FACS provides: 

(1) A list of all displays by name, which have been created, revised, or deleted 
since a specified date. 

(2) A list of all displays created by a given author. 

(3) A condensed version of a series of related displays. 

(4) A printed copy of ail or selected displays in the format in which they 
appear on the CRT screen, along with pictures of the displays as they appear in computer 
memory. Administrative information is also provided, such as name of the display, 
author, date last modified, and number of characters in the display. 

(5) A list of displays, by name, that contain a specified word or series. 

The FACS output has a variety of uses such as editing displays, scheduling course 
implementation activities, supervising the implementation team, and revising a course. 
Figure 16 shows an example of a FACS printout. 



EDITING 

Editing is facilitated by the combined use of FACS and EDITOR commands. 
Authors review FACS printouts of the instructional content and mark corrections on the 
hard copy. Through EDITOR commands, the original display can be called to the CRT 
screen and modified or deleted. Many such modifications may be made without changing 
the course organization or course programs. 



COURSE LOGIC 

Course programs contain the logic of a course— the rules for analyzing student 
responses, rules for making instructional decisions, and instructions on recording data. 

The programs are a series of instructions, usually written in the Coursewriter III 
author language (9). Coursewriter III is the special-purpose CAI language developed by 
IBM for use on the 360 computer (Coursewriter I and II were for the 1401 and the 1500 
computers). Project IMPACT has extended the capabilities of Coursewriter III to support 
CRT terminals and to make course programs more efficient. Hereafter this is referred to 
as IMPACT-Coursewnter. An example of a part of an IMPACT-Coursewriter program is 
presented in Figure 17. 

Embedded in the IMPACT-Coursewriter input-output instructions are commands to 
DIRECTOR, which tell DIRECTOR what CRT display text to retrieve from the text files 
and transmit to the student. Hence actual text is kept apart from the course programs. 
Coursewriter programs are ordinarily written on coding sheets and then keypunched. The 
punched cards are processed by a special Coursewriter program that assembles and stores 
the course program on disk. 

IMPACT-Coursewriter programs may also be input via a CRT or the 1050 type- 
writer. This procedure is used primarily for making short insertions, deletions, or 
modifications to a course program. 



ERIC 

•i'I’LT.TiTLi 







24 



Sample Portion of I IMPACT- Coursewriter Program 
DXP40 



1- 


0 


RD 


( (DIS D 365,1) .(SET GL0S=0)) 


(Command to DIRECTOR to transmit 


1- 


1 


EP 


di splay) 

(Command to poll student for response) 


1- 


2 


FN 


RECORD 


(Store a copy of the student's record) 


DBP22 


i _ 


0 QU 


((DIS D260,l ) ,(SET GL0S=0)) 


(Command to DIRECTOR to transmit 


1- 


1 


EP 


question) 

(Poll student for response) 


1- 


2 


ED 


CH*/CH*/CH* 


(Instructions for recognizing student's 


1- 


3 


LD 


DBP24/R1 


response) 

1 1 


1- 


4 


LD 


33/C69 


1 1 


1- 


5 


AA 


(L) WORKING*STORAGE* & 


1 1 


1- 


6 


LD 


1/SI 


1 1 


1- 


7 


LD 


4/BI ,52,1 


(Load decision factor valyes into 


1-1- 


8 


BR 


VAL02 


student's record) 
(Transfer control to IDM) 



Figure 17 



COURSE CHECKOUT AND DEBUG 

CAI courses, like any computer program, must be tested and debugged to ensure 
that the program is coded correctly and the displays properly synchronized with the 
logic. 

The separation of text and logic makes this debug operation more efficient than in 
standard Coursewriter systems. The IMPACT-Coursewriter programs may be debugged 
online, via 1050 or CRT, with only the titles of the displays being printed out; or the 
total course, including displays, may be tested via the CRT. The tester signs on the 
course as a student and tests various paths through the instruction. Via EDITOR and 
IMPACT-Coursewri'^r he can make corrections to either displays or logic directly on the 
terminal. 
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Part 4 



IMPACT-A HISSS CAPABILITIES 
FOR EVALUATING INSTRUCTION 



. BACKGROUND 

Ti 

jDne of the most significant characteristics of computer-administered instruction is 
the i^act that the instructional system has built into it the capability for collecting data 
needjd to evaluate its own effectiveness. The effectiveness of the instruction can be 
continually monitored and evaluated, and the instructional programs continually revised 
and improved. 

The computer’s powerful capabilities for collecting, storing, and manipulating data 
are widely recognized as being of tremendous potential value in helping to evaluate the 
instruction the computer administers. At the same time, if such data are to be usable for 
evaluation, the data must be managed , that is, stored, structured, extracted, summarized, 
and retrieved in terms that are meaningful for the purpose at hand. Data management, 
then, is an important function of the IMPACT HSSS. To the HSSS, the problem of 
evaluation is a problem of data management (Figure lc, “Evaluation Mode”). 



HSSS DATA MANAGEMENT CAPABILITIES 

The IMPACT-A HSSS data management capabilities are a combined function of the 
IMPACT-Course writer system’s capabilities for collecting and recording data, IMPACT 
Data Evaluation System’s (IDES) capabilities for structuring and retrieval, and the 
statistical analysis capabilities of the BMD package (10). An overview of these capabilities 
is shown in Figure 18. 



DATA COLLECTION 

Every time a student interacts with the computer, IMPACT-Coursewriter records 
information about that interaction. This feature is controlled so that an instructor or 
researcher may select the type of interaction to be recorded. For example, he may 
choose to have a recording made only when the student makes a mistake. 

Project IMPACT has added to the basic Coursewriter III recording mechanism the 
capability to specify the exact point in the instructional program at which a recording is 
to be made. This adds considerable flexibility and generality to the basic Coursewriter 
recording feature. Each time IMPACT-Coursewriter makes a student recording (Figure 
19), it stores 500 characters of data. 

IMPACT-Coursewriter writes these 500-character records on magnetic tape or disk, 
in the sequence in which the instructional interactions occurred. For example, a recording 
for Student 111 at Frame 63 may be followed by a recording for Student 112, at Frame 
14, followed by a recording for Student 554 at Frame 87 in another course. 
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IMPACT-A Data Management and Evaluation Overview 




Figure 18 
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Data Items in Student Performance Recordings 



• Student Name 

• Student Number 

• Group Number 

• Course Name 

• Course Segment 

• Date student registered 
•Last date student signed on 

• Current Date 

•Label identifying where student is in course 

• Last question student answered 
•Student's response to last question 

• Number of attempts at current question 
•Time student signed on 

• Current time 

• Total time student has spent on course 

• Decision factor values 

(6 registers, 81 counters, 32 switches, 

32 parameters) 

(100 characters of error codes) 

Figure 19 



IDES 



These sequential student recordings are the primary input to IDES, a set of 
computer programs that organizes the data into comprehensive structures, and provides 
for retrieval of selected and summarized information from the data structures (Figure 
20). Two data structures are produced— the Course Evaluation Data structure (CED) and 
the Student Evaluation Data structure (SED). At the time a day's response recordings are 
being structured by IDES, IDES also prints out a listing, by student, of responses and 
comments for the day. 



DATA STRUCTURES 

The CED is structured by IDES into lists of data cells (via list-processing subroutines 
called SLIP, 11) with the key item being a question or unit in the course. A schematic of 
the CED is shown in Figure 21. This data structure is designed to provide extracts, 
summaries, and analysis of performance across students within a course item. 

The SED is organized into SLIP lists with student identity being the key item. The 
SED contains not only the data provided by response recordings, but also biographical 
data, entry characteristics, and follow-up data obtained offline and entered via punched 
cards. A schematic of the SED is shown in Figure 22. 
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Overview of .IDES 




Figure 20 
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Course Evaluation Data Structure 
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Student Evaluation Data Structure 
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RETRIEVAL 



For course and IDM evaluation purposes, there are several types of reports that may 
be produced from the data structures. 

The comments and response listing shows, for each student, the actual text of his 
comments and responses in the order in which they occurred (Figure 23). It is extremely 
valuable to instructional programmers in checking on the effectiveness of individual items or 
units within the course. 



Sample Student Response Output 



STUDENT RESPONSE LISTING 



PAGE 1 



O 

ERIC 





STUDENT NO. 9997 


COURSE COBOL LEVEL 12 


DATE 3 JULY 1970 


Type materia! 


PRESENTATION FLAG 


STUDENT INPUT 


TIME 


Display (text) 


D1047 


001 THINK I UNDERSTAND 


14:17 


Question 


Q1050A NR 


MOVE A TO B 


14:19 


Feedback 


FI 1 2 


00WHY IS THE CONTENT OF 


A STILL THERE? 14:19 






Figure 23 




The 


unanticipated glossary 


requests listing shows words 


for which the students 



requested definitions that the system was not able to supply. It provides a mechanism for 
continual updating and improvement in the glossary and in the course material. 

The CEP printout of which Figure 24 is an example, is the primary tool for the 
instructional programmers in evaluating individual frames of instruction. 

The CED printout shows the state-of-kno\#ledge distribution across students on the 
item, the types of errors made by all students, and the requests for glossary information 
made by the students. Six “knowledge states” are provided for each attempt at answering 
the question. Knowledge states can be interpreted in any way desired by the course 
developers— for example, Project IMPACT’S COBOL course defines knowledge states as a 
function of the student’s expressed confidence in his answer, combined with the correct- 
ness of the response. 1 

Alternative uses might include a combination of aptitude classification with cor- 
rectness, a simple right-wrong categorization, or substantive classifications of correctness 
related to the subject matter. The report shows the number and percentage of students in 
each state of knowledge for each attempt at answering the question. (Some course 
strategies allow for the student to answer the same question several times until he gets it 
correct.) 

The error summary shown on the CED printout is extremely valuable in detecting 
the need for additional remediation sequences in the course, or for improving explana- 
tions of specific points. The error types may be defined in any way specified by course 
developers, and may be as refined as desired. Glossary references also provide quick 
information on the need for improvements in the course text on various points. 
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CED Output Format 



COURSE EVALUATION DATA FOR QUESTION Q1050A 
69/08/15 



STATE OF KNOWLEDGE DISTRIBUTION 



ATTEMPT NUMBER 



PAGE 21 



STATE 


F 


1 

P 


2 

F 


P 


F 


3 

P 


F 


4 

P 


5 

F 


p 


6 

F 


p 


7 

F 


p 


TOTAL 
F P 


WIF 


10 


50.0 


0 


0.0 


0 


0.0 


0 


0.0 


0 


0.0 


0 


0.0 


0 


0.0 


10 


33.3 


INF 


3 


15.0 


2 


33.3 


0 


0.0 


0 


0.0 


0 


0.0 


0 


0.0 


0 


0.0 


5 


16.7 


PIN 


3 


15.0 


1 


10.7 


0 


0.0 


1 


100.0 


0 


0.0 


0 


0.0 


0 


0.0 


5 


16.7 


UNI 


2 


10.0 


3 


50.0 


1 


33.3 


0 


0.0 


0 


0.0 


0 


0.0 


0 


0.0 


6 


20.0 


MIS 


1 


5.0 


0 


0.0 


1 


33.3 


0 


0.0 


0 


0.0 


0 


0.0 


0 


0.0 


2 


6.7 


HMI 


1 


5.0 


0 


0.0 


1 


33.3 


0 


0.0 


0 


0.0 


0 


0.0 


0 


0.0 


2 


16.7 


TOTAL 


20 


100.0 


6 


100.0 


3 


100.0 


1 


100.0 


0 


0.0 


0 


0.0 


0 


0.0 


30 


100. J 



MEAN POINTS 4.3 



6.2 



ERRORS FOUND IN STUDENTS ANSWERS 



ERROR 

NUMBER 




- ERROR TITLE 








6 9 7 


FORM 


VARIOUS 


EVALUATN 


8 8 8 


OTHER 


OTHER 


OTHER 


2 3 5 


ADD 


NAMES 


ANALYSIS 



GLOSSARY REFERENCES 
ITEM NAME 

COBOL 

FORTRAN 

READ 



NUMBER OF 
REFERENCES 
1 
1 
7 



NUMBER OF ERRORS ON ATTEMPT 
1 2 3 4 5 6 7 



2 1 

11 3 

0 1 



0 0 
0 0 
0 0 



F — Frequency of students 
P — Percentoge of students 
WIF = Well Informed 
INF “ Informed 



PIN — Portiolly informed 

UNI = Uninformed 

Ml 5 = Mi sinformed 

HMI = Highly Misinformed 



Figure 24 
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SEP Retrieval is highly dependent on the requirements of a particular course or 
IOM evaluation study. The SED is so comprehensive that each evaluation study must 
specify the exact extraction, summary, and analysis needed for that oarticular study 
(Figure 22). 

Retrieval functions, a part of the IDES package, assist in extracting the specific data 
items needed from the SED. These items may be printed out either in detailed or in 
summary fashion, (e.g., a listing of an individual’s grades on quizzes and tests) or they 
may be processed by statistical packages. At Project IMPACT, the BMD statistical analysis 
package (10) is used to produce various statistical reports from SED data (Figure 25). 

A wide range of evaluation studies can be performed using data in the SED in 
combination with the BMD statistical analysis package. For example, response measures 
(latency, particular errors, patterns of responding, expectations by the student, and so 
on) can be related to pre-course student history, within-course student history, achieve- 
ment criterion measures, and other factors in the IDM used to make instructional 
decisions. 

The variety of techniques available in the BMD analysis system makes it possible to 
explore joint effects of several variables (multivariate analysis), the effects of qualitative 
or categorical variables (contingency analysis), mathematical formulations of stages of 
learning (time series analysis), the unique pattemings of students’ characteristics (factor 



41 i 



33 



List of Statistical Analyses Performed by BMD 



Description and Tabulation 

Simple Data Description 
Correlation with Transgeneration 
Correlation with Item Deletion 
Alphanumeric Frequency Count 
General Plot Including Histogram 
Description of Strata 
Description of Strata with Histograms 
Cross-Tabulation with Variable Stacking 
Cross-Tabulation, Incomplete Data 
Data Patterns for Dichotomies 
Data Patterns for Polychotomies 

Multivariate Analysis 

Principal Component Analysis 
Regression on Principal Components 
Factor Analysis 

Discriminant Analysis for Two Groups 
Discriminant Analysis for Several Groups 
Canonical Analysis 
Stepwise Discriminant Analysis 

Regression Analysis 

Simple Linear Regression 
Stepwise Regression 

Multiple Regression with Case Combinations 
Period Regression and Harmonic Analysis 
Polynomial Regression 
Asympototic Regression 



Time Series Analysis 

Amplitude and Phase Analysis 
Autocovariance and Power Spectral Analysis 

Variance Analysis 

Analysis for Variance for One-Way Design 
Analysis of Variance for Factorial Design 
Analysis oi Covariance for Factorial Design 
Analysis or Covariance with Multiple Covariates 
General Linear Hypothesis 
General Linear Hypothesis with Contrasts 
Multiple Range Tests 
Analysis of Variance 

Special Programs 

Life Table and Survival Rate 
Contingency Table Analysis 
Biological Assay: Probit Analysis 
Guttman Scale Preprocessor 
Guttman Scale #1 
Guttman Scale #2, Part 1 
Guttman Scale #2, Part 2 
Guttman Scale #2, Part 3 
T ransgeneration 

Transposition of Large Matrices 



SOURCE: Reference 10. 



Figure 25 



analysis), relationships between individual characteristics of the student such es expect- 
ancies in relation to actual performance (correlational and regression analysis). The BMD 
analysis system will be extended in Project IMPACT to increase the precision of esti- 
mating stages in the learning process (Bayesian and conditional probability analyses); 
these additions to BMD will allow use of formal mathematical models of the learning 
process. 

Taken as a whole, evaluation studies provide the mechanism for a continuous 
refinement of IDM and overall efficiency of the IMPACT computer-administered instruc- 
tional system. 
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Part 5 



IMPACT- A HSSS CAPABILITIES 
FOR SUPPORTING SCHOOL ADMINISTRATION 



Although not explicitly designed as an administrative record keeping system, the 
IMPACT-A HSSS, in conjunction with the instructional process, keeps records that are 
useful for administrative purposes. A fully operational CAI system will be designed to 
interface smoothly and directly with school administration and operations. The capabili- 
ties described here are a step toward that objective (see Figure Id, “Support Mode”). 



REGISTRATION 

Students are registered for a course through a course supervisor who enters special 
commands into the computer. One student or a group of students may be registered in 
one command, either through the 1050 typewriter or the CRT. When a student is registered 
for a course, the IMPACT-Coursewriter administrative software creates an identification 
record for him, which the system continuously updates as the student progresses through 
the course. Some of the items of information in that record are shown in Figure 26. 



Information Items in 
Student Identification Record 

• Course Name 

• Course Segment 

• Student Name 

• Date Registered 

• Last date student was using course 

• Student’s Group number 

• Student's Identification number 
e Where student is in the course 



• Current date 

• Time of day student last signed on 

to course 

• Total time, in minutes, student has 

spent on course to date 

• Student's response to last question 

• Performance date-status of counters 

and switches and error codes 



Figure 26 



INQUIRIES 

A supervisor or instructor may inquire as to the status of a student (9). At the CRT 
or 1050 typewriter, he keys in the inquiry using the Supervisor's Command Language. 
The status report tells where the student is in the course, how long he has been on the 
course (actual minutes of instruction), when he was last on the course, and the date on 
which he started it (Figure 27). 
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Example of System Response to Student Status Inquiry 



GROUP COURSE LOCATION START DATE LAST DATE TIME TYREC STUDENT NO. STUDENT NAME 



RJS 


DAR2 


0- 


1 


6/5/70 


6/24/70 


0:37 


0 


19 70 


GROUP 1 


RJS 


F0034 


0- 


2 


6/15/70 


6/16/70 


1: 8 


0 


1971 


GROUP 2 


RJS 


DAP Id 


1- 


1 


6/11/70 


6/26/ 70 


1:10 


0 


19 72 


GROUP 3 


RJS 


TEMPI 


1- 


1 


6/15/70 


7/13/70 


9:41 


0 


11001 


NEIL ZAREMBA 


RJS 


DAC60 


1- 


1 


6/22/70 


V26/70 


10:13 


0 


11002 


CHARLES JONES 


RJS 


HMS 


1- 


1 


6/23/70 


6/26/70 


8:13 


0 


11003 


RICHARD CAMPBELL 


RJS 


PLACE 


0- 


2 


7/ 2/70 


7/10/70 


13:17 


0 


11004 


DOUG SHAW 


RJS 


TEMPI 


1- 


1 


6/30/ 70 


7/ 9/70 


7:26 


0 


11005 


DOUG SPENCER 


RJS 


D2131 


1“ 


1 


7/ 7/70 


7/10/70 


11:57 


0 


11007 


GARY WOLFE 


RJS 


Q2691A 


1- 


1 


7/13/70 


7/17/70 


13:14 


0 


11008 


DAVID K0PSTEIN 


RJS 


D2435Z 


1- 


1 


7/13/ '’O 


7/17/70 


13: 63 


0 


11009 


JEFFREY SEWARD 


RJS 


D1380 


1- 


1 


7/13/70 


7/16/70 


4:15 


0 


1944 


DOUG ROBINS 



Figure 27 



The Project IMPACT staff is now designing computer software algorithms that will 
predict the date at which a student will complete the course, based on his performance 
to date. This prediction would be added to the status report. 

In a CAI system with remote student stations in different buildings or geographic 
locations it is useful to be able to locate a student or determine the status of the 
stations. An inquiry can be made that tells a supervisor or instructor who is using which 
stations at a given point in time. 



ACADEMIC RECORDS 

Grades on quizzes, exercises, phase tests, or other tests are stored by the system in 
the student record files. Attendance data is also recorded, showing date, time of day, and 
duration of each student session. These data may, with some modification in the present 
SED structure, be stored and retrieved in various detailed or summary reports for 
academic record keeping (see Part 4). Such data may be used by the instructor for his 
record keeping, or may be used directly by the school administrative office for academic 
records. In schools where the Academic Records Application of the CONARC Educational 
Data System (CONEDS) has been implemented, IMPACT-B would provide compatible 
data interface with CONEDS (12). 
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Part 6 



OPERATIONAL SUMMARY 



It is commonly recognized that if a CAI system is to be not only effective as an 
instructional method, but also efficient and economical for use by Army schools, the 
system must have a very large operating capacity. The larger the system in terms of 
numters of students it can accommodate and numbers of courses available, the more 
economical it is on a per student basis (13). 

With this premise in mind, development of the IMPACT CAI system began with a 
medium-large scale computer that is typical of the kind that would be required in a large 
scale CAI, system. The CRT device is of the general type that could be used in large 
numbers and at locations remote from the central computer. 

From an operations standpoint, the IMPACT-A HSSS can be described as evolu- 
tionary. That is, although some characteristics of the hardware and computer software are 
experimental in nature, they are designed to provide a basis for evolution to a large scale, 
efficiently operating system. For example, the Sanders CRT, which is currently an 
important part of the system, is a general-purpose alphanumeric device, with many varied 
and flexible capabilities. Project IMPACT will determine which of these capabilities are 
really needed for implementation of effective operational CAI. The result of this study 
and experience will be design recommendations for a CRT that is tailored to Army CAI 
requirements. 



HARDWARE SUMMARY 



The central processing computer is a moderate-size third-generation computer, the 
IBM 360 Model 40 (14). It has 256,000 bytes of internal storage. Auxiliary storage is 
provided by an IBM 2314 direct access storage device, containing eight disk drives (15), 
each drive having a capacity of 23,000,000 bytes. There are four 2401-IV (120KB) 
magnetic tape drives. Input and output to student stations is provided through the 
Sanders 731 Communication Buffer (for CRTs) or the IBM 2701 Data Adapter (for 
1050s 7 ). Telecommunications lines connect the Sanders 731 to the Sanders 701 Control 
Units located near the CRT terminals. Each 701 controls three Sanders 720 CRTs (16) 
and one teletypewriter for hard copy of information on the screen. (The IMPACT-A 
hardware configuration is shown in Figure 28.) 

Various capabilities of the IMPACT student stations have been mentioned through- 
out this report. The functions of the Sanders 720 Terminal (viewing, presentation, 
format, and control) are described in Figure 29. 



7 The 1050 terminal configuration was deleted from the system, for budgetary reasons, as of 
1 July 1970. 
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IMPACT A Hardware Configuration 




Inlormotion Pfoc»»«mg Sub»y»t»m 
Doio Sloiag* Sub«y»t»n> 
Communication Subtyttom 



Figure 28 
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Sanders 720 Terminal Summary 



Function 



Description 



Viewing 



Presentation 



Format 



Control 



Screen color grey. Adjustments for horizontal, vertical, and 
brightness. The screen size is 7 V 2 by W 2 inches. 

Of the screen's 2,048 character positions, 1,024 may be used 
for information to be presented and control characters. 

The screen may be divided into a number of individually 
addressable blocks. The character positions forming a 
block need not be contiguous. 

Emphasis is obtained by use of the blinking feature in which 
individual characters or words may be caused to blink. 

Extensive formatting capability is available through the use 
of edit control characters for horizontal and vertical 
spacing, cursor positioning, etc. 

Control over student using the keyboard is accomplished 
through a hardware feature that provides two modes of 
operation. In the student (Edit-1) mode, the student 
cannot destroy the presentation format. 

The student can be prevented from moving the cursor outside 
the block he is currently working from the keyboard. 

The light pen allows the cursor to be moved to any point on 
the screen. 



Figure 29 



SOFTWARE SUMMARY 



When CAI is operating with student stations, for administering courses, or for author 
use, the modules used are: IMPACT-Coursewriter, Zeus, DIRECTOR, EDITOR, and 
Operating System/360. IDES and FACS operate offline— that is, independently of Course- 
writer, Zeus, and the student stations. (The components and functions of the computer 
software subsystem are reviewed in Figure 30.) Each of the software components is 
identified in Figure 31 in terms of its origin, dependencies and status. 
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IMPACT Computer Software Configuration 

ONLINE OPERATIONS OFFLINE OPERATIONS 
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Summary of Software Components— 
Dependencies, Status, and Availability 



Component 




Hardware 


Software 




Name 


Origin 


Dependencies 


Dependencies 


Status 


360 

Operating 

System 


IBM 


360/40 


None 


Latest version 


Coursewriter 

III 

Version 1 


IBM 


IBM 360 
1050 


OS/360 


Released, type III 
documented not 
supported by 
IBM 


IMPACT- Coursewriter 


IMPACT 


IBM 360 


OS/360 


Operational, 

documented 


IMPACT- Coursewriter 
functions 


IMPACT 


IBM 360 


IMPACT- 

Coursewriter 


Operational, 
documented for 
users and pro- 
grammers 


Zeus Supervisor® 


IMPACT 


IBM 360 

Sanders 720 CRT 


OS/ 360 


Operational, 

evolving 


EDITOR 

Commands 


IMPACT 


IBM 360 
CRT 


OS/360 

Zeus 


Operational 


DIRECTOR 


IMPACT 


IBM 360 
CRT 


OS 366 
Zeus 


Operational, 

evolving 


IDES 


IMPACT 


IBM 360 


Fortran IV 
IMPACT- 
Coursewriter 
SLIP 


Operational, 

documented 


FACS 


IMPACT 


Sanders 720 CRT 
IBM 360 


PL-1 

Zeus 


Operational 



°Not available for release at this time. 



Figure 31 




OPERATIONAL CHARACTERISTICS 
MODES OF OPERATION 

Actual instruction and course preparation activities may take place simultaneously, 
that is, students may be receiving instruction on some display consoles while courses are 
being formatted and coded on others. A number of different courses may be adminis- 
tered simultaneously. The system allows multiple applications to share the main 
computer, that is, IMPACT and unrelated jobs may be conducted at the same time (for 
example, administrative record keeping and other computational batch processing jobr). 
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CORE REQUIREMENTS 



The IMPACT software system requires approximately 100,000 bytes of core (about 
one-half of the total available for applications, since the IBM 360 Operating System 
requires approximately 50,000). 



NUMBER OF STUDENTS 

The number of students that can be handled at one time is a function of the 
number of student stations that the system has online and the number of students that 
can share a terminal. 

Number of Student Stations . There is no theoretical limit to the number of CRT 
terminals that can be connected to the computer. However, there is a practical maximum, 
with the response time as the constraining factor. The greater the number of terminals on 
a line, the slower will be the response time as the waiting time increases. 

The number of student stations supported in the system is a critical factor for 
operational CAI. This will be a major consideration in IMPACT-B design. 

Registration Limit . Up to 999 students may be registered for a given course at any 
point in time. 

Location of Student Stations . The terminals can be either local (close to the 
computer) or at remote sites via telecommunication lines. 



DIRECT ACCESS STORAGE 

Instructional Text. The amount of course material that may be available to students 
at any point in time depends on the amount of direct access storage available. The 
IMPACT configuration has eight disk packs on the IBM 2314. The capacity of each disk 
pack is 23,000,000 bytes. (Each pack has 230 times the storage of one disk cartridge on 
the IBM 1500.) About 20,000 full pages of CRT text can be stored on one of these disk 
packs. 

Hours of Instruction. The translation from bytes of storage to hours of instruction is 
not a direct one. There are a great many variables affecting the hours of instruction 
provided so that one particular hour of instruction may require 10 or 20 times more disk 
storage than another hour of instruction. However, it is estimated that several thousand 
hours of instructional material may be stored at one time on the IMPACT-A 
configuration. 

Instructional Program, Disk storage is also required for the course logic (Course- 
writer program and IDM). Each instruction within the program typically requires from 15 
to 20 bytes of storage. One hour’s worth of instruction requires anywhere from 700 to 
1,000 program instructions. Therefore a thousand hours of instruction would require 
from 10,500,000 to 20,000,000 bytes of storage for the Coursewriter program and IDM. 
This is less than one 2314 disk pack. 



RESPONSE TIME 

The response time is defined as the time interval between tne instant a student 
presses the “Send” key jn the keyboard to the instant the initial response from the 
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computer is projected to him. The primary factors within the HSSS that affect response 
times are: 

(1) Number of students online at any one time. 

(2) Number of computer interactions with each student per unit time. 

(3) The rate at which data is transmitted. 

(4) The length of student responses. 

(5) The computer program time to analyze the response, and make instruc- 
tional decisions. 

(6) Time required to access disk files. 

The response time for a given IMPACT configuration may be predicted through 
simulation. 



OTHER OPERATIONAL CONSIDERATIONS 

On the basis of experience with IMPACT-A HSSS, the requirements for and design 
of the next generation prototype (IMPACT-B) will be prepared. The IMPACT-B HSSS 
will include provisions for some functions and capabilities needed in an operational Army 
environment that are not a part of the IMPACT-A system. These are: 

(1) Security . The handling of classified instructional materials, not within the 
IMPACT-A system capabilities, will be addressed in the requirements analysis for 
IMPACT-B. 

(2) Fallback and R ecovery . The ability to return to a minimum operation 
when certain system components fail, and to recover from disaster conditions, will be 
addressed in IMPACT-B requirements and specifications. 

(3) Reliability . The minimum standards for reliability of individual components 
and the total system, in an operational environment, will be a part of the IMPACT-B 
specifications. 

(4) Interface With CQNEDS . The Academic Records Application of the 
computer-based CON ARC Educational Data System is currently operational in some 
Army schools (12). IMPACT-B will be designed to provide a compatible data interface 
with the Academic Records Application. 
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Appendix A 

GLOSSARY 



CAI 


Computer-Administered Instruction. An individualized system of 
instruction in which the computer presents instruction to a student 
on the basis of his progress and other characteristics. 


CED 


Course Evaluation Data. A file containing summary data on student 
performance in a CAI course of instruction; used to evaluate and 
improve courses of instruction. The CED is structured and managed 
by IDES and is stored in computer-readable form on magnetic disk. 


COBOL 


Common Business Oriented Language. A standard computer pro- 
gramming language use'' 3 by the Army for nearly all nonscientific 
computer applications such as supply, accounting, and personnel. 


Country 


Computer storage spaces used to store data on a student’s perform- 
ance during a course. 


Coursewriter III 


A computer software package developed by IBM for instructional 
applications on the IBM 360 computer. 


CRT 


Cathode Ray Tube. This device is similar to a television screen, and 
is used to communicate from computer to student. The CRT has an 
associated typewriter-like keyboard through which the student 
communicates wiuh the computer. 


DIRECTOR 


A software supervisor that retrieves the appropriate course content 
for an individual student and transmits it to his station. It is in inter- 
face between course programs and IDM, the course materials, and the 
student. 


Display 

EDITOR 


Any material projected onto a CRT screen. 

ISntry on Disk of ^Instructional Text for Online Retrieval. A set of 
commands that enables course authors to create and modify course 
materials on the CRT and have them stored on disk at the central 
computer. 


FACS 


File Activity Control System. A set of computer programs that pro- 
vides information about the display pages that have been stored on 
disk and helps in editing and coordinating ihe displays. 


HSSS 


Hardware/Software Subsystem. A subsystem of a twtal CAI system. 
The total system includes hardware, computer software, instructional 
materials, rules for administering the instruction, and students. 


IDES 


IMPACT Data Evaluation System. A set of computer programs that 
manages the data collected, stored, and processed by the CAI system. 
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IDM Instructional Decision Model. A set of rules for determining what 

instruction to present next to a student, based on a combination of 
such factors as his performance during the course, his aptitude test 
scores, and his requests for assistance. 

IMPACT Instructional Model Prototypes Attainable in Computerized Training. 

The advanced development project in CAI at HumRRO. 

IMPACT- The Coursewriter III software package as used at Project IMPACT. 

Coursewriter The major difference between IMP ACT-Course writer and Course- 

writer III is the addition of the capability to use CRT terminals 
as the interface with the student. 

Modes Author Mode. When instructors or researchers are interacting with the 

computer, preparing instructional text or logic, the system is said to 
be in author moae. 

Studenb Mode. When the computer and student are interacting dur- 
ing the course of instruction, the system is said to be in student mode. 

Evaluation Mode. When the computer system is preparing reports or 
statistical analyses of student performance for research and evalua- 
tion purposes, it is said to be in evaluation mode. 

Support Mode. When members of the school or CAI center interact 
with the computer to get administrative information about students 
or other aspects of the system, it is said to be used in support mode. 

Offline Any process, activity, or device that is not online in the interactive 

sense defined below. Offline processes may be performed by man, 
computer, or other devices. 

Online A person or machine interacting directly with a computer, usuallv 

over a communications line; also, a process or activity pe>fr ^d ir 
or with the computer, while the computer interacts with arsons or 
machines through a communications line. The process or activity 
must affect the computer-man interaction in order to be considered 
online (e.g., batch operations such as FACS operating in background 
mode, even though they physically share the computer with online 
operations, are not considered to be online ). 1 

Page All the material projected at one time on a CRT screen. 

SED Student Evaluation Data. A file containing both detailed and 

Summary - data on students and their performance in CAI courses 
of instruction; used to evaluate individual and groups of students, 
courses of instruction, and instructional decision models. The SED 
is structured and managed by IDES and is stored in computer read- 
able form on magnetic disk. 

SEEK A computer software function developed at Project IMPACT to 

assist in the implementation of instructional programs. 



1 There are more narrow definitions of online in terms of input-output hardware connected 
directly to a computer; however, they are not useful in distinguishing between processing modes. 
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SLIP 

SOK 

Zeus 



A collection of computer software subroutines used in computer 
programs that manipulate complex lists of data items. 

State of Knowledge. A way of categorizing a student’s knowledge 
or skill based on his actual performance of a task and on his own 
confidence in his ability to perform the task. 

A computer software monitor that makes it possible to operate 
multiple CRT terminals from locations remote from the central 
computer. 
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KEY TO FLOWCHART SYMBOLS 



Processing 



Input/ 

Output 



Magnetic 

Tape 




Preparation 




Punched 

Card 



Printout 




Online 

Storage 



Auxiliary 

Operation 
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US ARMY TROPIC TEST CTR PO DRAWER 942 ATTN BEHAV FCIEN CZ 
CG III CORPS L FT HOOD ATTN G3 SEC FT HOOD 
CO 1ST ARMORED OIV ATTN G3 SEC FT HDDO 
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30 CG 2D ARMORfO Dl V ATTN G3 SEC FT HnPD 
25 CO 13TH SOPT BGDE ATTN S3 SEC FT HOTO 
10 CG USAFAC 6TTN G3 SEC fT SILL 

20 co m rrmps artv attn 03 sec ft sill 

20 CG USA AD CTR ATTN G3 SEC FT BLISS 
3 CG ATTN 03 SLC FT POLK LA 
I 0ESO A«n OFC CHF OF TlCO MASH DC 

1 CHF OF REO DA ATTN SCI INFO 8R RSCH SPI DIV WASH DC 

2 C INC US PACIFIC FIT FPD 96614 SAN FRAN 

} ATlANTIC FLT CODE 3L2 A USN BASE NOPFOLK 

l COR TNG COMMAND US PACIFIC Ft T SAN DIEGO 
5 TECH UR PEPS 1 1 fl BUR OF WAV PERS AflL ANNEX 

3 DIR PERS RFS OIV BUR OF NAV PERS 

1 TECH LIB BUR OF SHIPS CODE 2I0L NAVY DEPT 
[ HUMAN FACTORS OR PSYCHOL «ES D:v ONR 
l FNGNR PSYCHOL SR ONR CnOE A** ATTN ASST HF AD WAS’I DC 
3 CO *■ DIR NAV TNG OEVtCE CTft jRLANDO ATTN TECH LlB 
l CO FLT ANTI-AIR WARFARE TNG SAN OIEGO 

1 CO NUCLEAR WEAPONS TNG CTR PACIFEC U S NAV A I R STA SAN DIEGO 

2 CO FLT TNG CT» NAV BASE NEWPORT 

1 CO FLEET TNG CTR U S NAY STA SAN OIEGO 
1 CLIN PS'/CHOL MENTAL HYGIENE UNIT US NAV ACAD ANNAPOLIS 
1 PRtS NAV WAR COLL NEWPORT ATTN MAHAN LIB 

J t. 5. ,P ftTLANT,C FLT ANTt-SUR WARFARE TACTICAL SCH NORFOLK 

1 CO NUCLEAP WEAPONS TNG CTR ATLANTIC NAV A 1 R STA NORFOLK 

2 CO FLT SC1NAR SCH KEY WEST 

1 CO FLT ANTI-SUB WARFAPF SCH SAN OIEGO 
1 CHF OF nav RES ATT IV SPEC ASST FOR A C 0 

1 CHF OF NAV RES ATTN HCAO PERS ♦ TNG 0R CODE A 56 

l CHF OF NAV PES ATTN HFAO CP PSYCHOL BR COD? 452 

L 01 R US NAV RES LAb ATTN CODE 5170 

L OIR NAVAL RSCH ATTn LIB CODE 2029 ( DNRL ) WASH DC 
l CHF OF NAV AIR TNG TNG RES DEPT NAV AIR STA PENSACOLA 

l CO MFD FLO RES LAB CAMP I.EJEUNE 

1 COR NAV MSL CTP POINT MUSI/ CALIF ATTN TECH LIB CG OE 3022 
l OIR AEROSPACE CREW EQUIP LAB NAV AIR FMGNR CTR PA 
1 OIC NAV PER S PFS ACT VY 5 AN DIEGO 

1 NAV NEUROPSYCH] AT RFS UNIT SAN DIEGO 

2 NAVAL MSL CTR (CODE 53521 PT HUGU CALIF 

l DIR PERS RES LAB NAV PERS PROGRAM SUPPORT ACTIVITY WASH NAV YO 

* NAV TNG PCRS CTR NAV STA NAV YD ANNEX CODE 63 ATTN LlB WASH 

1 COHDT MARINE CORPS HO MARINE CORPS ATTN CODE A 0-18 
1 HQ MARINE CORPS ATTN AX 

1 DIR MARINE CORPS EDUC C1R MARINE CORPS SCH QUANTICO 

1 DIR MARINE CDRPS INST A1TN EVAL UNIT 

1 CHF OF NAV OPNS OP-OIP1 
I CHF Of NAVL OPS OP 037 HASH OC 

1 CHF OF NAV OPNS 0P-07T2 

2 COMDT HOS RTH NAV 01 ST MTIN EDUC AOV NEW ORLEANS 
L CHF OF NAV AIR TECH TNG NAV AIR STA MEMPHIS 

L DIP. DPS EVAL GRP OFF OF CHF OF NAV OPS 0PH3EG 
2 COMOT PTP CDAST GUARD HO 

L CHF OFCP PERS RES + REVIEW BP COAST GUARO HQ 

1 CO US COAST GUARO TNG CTR GOVERNORS ISLAND NY 

1 CO US COAST GUARD TNG CTft CAPE MAY NJ 

1 CO US COAST GUARD TNG CTR £ SUP CTP ALAMEDA CALIF 

1 CO US COAST GUARD INST OKLA CITY OK LA 

1 CO US COAST GUARD RES TNG CTP YORKTOWN VA 

1 SUPT US COAST GUARD ACAO NEW LONDON CONN 

1 OPNS ANLS OFC HQ STRATEGIC ATR COMD O e F UTT AFP 

1 AIR TNG COMO RANDOLPH AF 0 ATTN ATFTM 

1 TECH DIR TECH TNG OIVIKRD) AFHRL LOWRY AF 0 COLO 

1 CHF SCI DIV DRCTE SCI + TECH OCS P+D HO AIR FORCE AFRSTA 

I CHF OF PEPS RES RR ORCTE OF CIVILIAN PF4S OCS-PERS HQ AIR FDRCE 

1 CHF ANAL DIV (AFPOPL (ID DIR OF PERSONNEL PLANNTNG HQS USAF 

1 HtJ AFSC SCAB ANOREVS AFB 

2 COR ELEC SYS OIV L G HJNSCOM FlO ATTN ESRHA BE OF FRO MASS 
HQ SAMSO ISMSIR) AF UNIT POST OFC LA AFS CALIF 

2 KILIT TNG CTR 0P5 LACKLANO AFB 
2 AFHRL (HRTI HR IGHT-P ATTERSDN AFB 
I AMD AHRH BROOK AFB TCXAS 

1 HOS ATC DCS/TECH TNG fATTMSI RANDOLPH AFB 
A HOS ATC IATCTD-MI RANDOLPH AFB TEXAS 
L COR ELEC SYS OIV LG H ANGCDH FLO ATTN ESTI 
i OIR AIR U LIB M4XWELL APR ATTN CUL3T-63-253 
1 OIR OF KB US AIR FORCF ACAO 

] COHDT OEF WPNS SYS MGT CTR AF INST OF TECH WR !GHT-P ATTERSON AFB 

I COMOT ATTN LIB OEF WPNS SYS ^GT CTR AF INST OF TECH WR I GHT-P AT* 

1 65/OTH PERS RES LAB PHA-4 AEPOSPACE MED OTV LACKLAND AFB 

1 TECH TNG CTR ( LMTCYOP-I-LL I LOWRY AFB 

2 AF HUMAN RESOURCES L All MRHTO WP. I GHT-P AT TER SDN AFB 

2 CO HUMAN RF SOURCES LA) BROOKS AFB 

L PSYCHOBIOLOGY PROG NATL SCI FOUND 

I OTP NATL SECUP A GY FT GEO G MEA06 attn TDL 

l 0I« NATL SECUR A GY FT GED G MEAOE ATTN OIR DF TNG 

5 CIA ATTN OCR/ADD STANDARD OTST 
1 SYS EVAL OIV RES DIRECTORATE OOO-DCD PENTAGON 
l OEPT OF STATE BUR OF INTEL ♦- RES EXTERNAL RES STAFF 

1 SCI INFO EXCM WASHINGTON 

2 CHF HGT C GEN TNG DIV TR 200 FAA WASH DC 

L BUR OF RFS £ ENGR US POST OFC DEPT ATTN CHF HUMAN FACTORS BR 

1 EOUC HEOIA BR OE OEPT OF HEW ATTN V D CLEMENS 

L OFC OF INTERNATL TfIG PLANNING C EVAL BR AID WASH DC 
L DEPT OF TRANS FAA ACC SEC HQ 6lOA WAi.H DC 
L SYS OEVEL CORP SANTA MONICA ATTN LlB 

2 DUNLAP ASSOC INC OARIEN ATTN LIB 
2 RAC ATTN LIB MCLEAN VA 

1 RAND CDRP WASHINGTON ATTN LIB 

1 DIR RANO CORP SANTA MONICA ATTN LIB 

2 U OF SO CALIF elec PEPS RES GP 

l COLUMBIA U ELEC RES LABS ATTN TECH EDITOR 

1 MITRE CORP BEDFORD MASS ATTN LIB 

2 SIMULATION ENGR CORP ATTN DIR OF ENGR FAIRFAX VA 
2 U OF PGH LEARNING RfII CT' ATTN OIR 

I HUMAN SCI RES INC MCLEAN 'A 



2 TECH INFO CTR ENGNR DATA $FRV N AHER AVN INC COLUMBUS D 
l Chrysler corp hsl div Detroit attn tech info ctr 

1 RAYTHEON SERV CO ATTN LIBN BURLINGTON HASS 

2 EDUC L TNG CONSULTANTS ATTN l C SILVERN LA 

1 GEN DYNAMICS POMONA OIV ATTN LIB DIV CAL I r 

2 harquardt jndsir prod cd cucamonga calif 

1 OTIS ELEVATOR CO OIV ATTN LIB STAMFORD CONN 

\ rrS fllOTECHNOLOGY AEROSPACE SVS OIV MS 0H-25 BOEING CO SEATTLE 

2 CTR FOR RES IN SOCIAL SYS FLO OFC FT BRAGG 
I IDA RSCH £ FNC, SUPT OIV ARL VA 

1 HUGHES AIRCRAFT COMPANY CULVER CITY CALIF 
1 OTR CTR FOR RES ON LEARNING ♦ TEACHING U OF MICH 
1 EDITOR TNG RES ABSTR AHER SCC OF TNG OJ RS U OF TENN 
1 CTR FOR RES IN SOCIAL SYS AMFR U 
-V BRITISH 6MRSY BRITISH DEF RES STAFF WASHINGTON 

3 CANADIAN JOINT STAFF OFC OF OEF RES MEMBER WASHINGTON 
3 CANADIAN ARMY STAFF WASHINGTON ATTN GS02 TNG 

? ,NI ! L F0REIGN LIAISON OFCR TO NORM EG HlLlT ATTACHE 

1 APMY ATTACHE ROYAL SWEDISH EMBSY WASHINGTON 
l DEF RES ME 0 lYB ONTARIO 

3 AUSTRALIAN NAV ATTACHE EH0SY DF AUSTRALIA WASH DC 

1 OFC OF A I R ATTACHE AUSTRALIAN EM6SY ATTN: T.A. NAVGN WASH» D.C. 

2 AUSTRALIAN EMBSY OFC OF MILIT ATTACHE WASHINGTON 

2 U O'- SHEFFIELD DEPT OF PSYCHDL 
1 Mt NN1NGER FOUNDATION TDPEKA 

1 AHER INST FOR RES SILVER SPRING 

1 ANER INST FOP RES PGH ATTN LIBN 

1 DIR PRIMATE LAB UN I V OF HIS MADISON 

3 MATRIX CHRP ALEXANDRIA ATTN TECH LIBN 
1 AHER TEL+TEL CD NY 

1 U OF GFOPGIA OEPT OF PSYCHOL 

L DR GEORGE T HAUTY C HHN DEPT DF PSYCHOL U OF DEL 
1 VITRO LABS SILVER SPRING MD ATTN LtBN 
l HEAD DF PT OF PSYCHni UNIV DF SC COLUMBIA 

1 TVA ATTN CHF LABOR RELATIONS BR OIV OF PERS KNOXVILLE 

1 U OF GEORGIA DEPT OF PSYCHDL 
l GE CO WASH D C 

l AHER INST FDR RES PALO ALTO CALIF 
1 MICH STATE U COLL OF SOC SCI 
1 N MEX STATE U ATTN PROF OF PSYCHOL 
1 ROWLANO •*- CO HADDONFIELD NJ ATTN PRES 
I OHIO STATE U SCH DF 4vN 

1 SCI RSCH ASSOC INC DIR OF EVAL CHICAGO ILL 

1 AIRCRAFT ARMAMENTS INC CDCKEYSVILLE HO 

2 OREGON STATE U DEPT DF MILIT SCI ATTN ADj 
1 TUFTS U HUMAN ENGNR INFO «■ ANLS PRDJ 

I AMER PSYCHCl ASSOC WASHINGTON ATTN PSYCHOL ABSTR 
L NO ILL U HEAD DEPT OF PSYCHDL 

f u»^twr!n/r! RCHrLD HILLER REPUBLIC AVN OIV FARMINGDALE N Y 
1 WASHlNGTnN ENGNR SERV CO INC KENSINGTON HO 
1 LIFE SCI INC FT WORTH ATTN PRES 
1 AMER 8 EHA V SCI CALIF 
1 COLL OF MM ;• MARY SCH OF EDUC 
I SO ILLINOIS U OF PT OF PSYCHOL 

1 CT ” DEVEL * C0 NSUL TATtON SERV SECT ATLANTA 

2 WASH MILITARY SYS OIV BETHFSOA MD 

1 NORTHWESTERN U DEPT OF I MOST R ENGNR 
1 HONEYWELL ORO STA MAIL STA 606 MINN 
I NY STATE EOUC DEPT ABSTRACT EDITOR AVCR 
1 AEROSPACE SAFETY OIV U OF SOUTHERN CALIF LA 
] mr BRANDON B SMITH RES ASSOC U OF MINN 
1 CTP FOR THE ADVANCED STUDY OF EOUC AOMIN U OF GREG 
1 OR V 2ACHFRT RT 2 NORMAN PARK G4 
1 J P LYDON DIR JR ROTC SAN ANTONIO TI£X AS 
I OR E FOULKE DEPT np PSYCH UNIV OF LOUISVILLE 
1 DR E PERKINS PROF DF PSYCH ST CLOUD STATE CDLL HlNN 
1 MR S AllES STEPTOE C JOHNSON WASH OC 

I DR W BEVAN VP l PROVOST THE JOHNS HOPKINS UNlV MO 

l DR W C BIEL U DF SOUTHERN CALI F LA 

L OR C W 3PAY BOX A2A QUOGUE LI W 

1 MR J M CHRISTIE PRES RIGGS NATL BANK WASH DC 

L OR C W CLARK V=> FDn RSCH RSCH TRIANGLE INST NC 
1 GFN H P HARRIS (USA RETIPREi THE CITADEL SC 
1 OR L T RAOEft CHHN DEPT OF ELEC ENGNR U OF VA 
1 CHF PPOCFSSING OIV DUKE U 110 
1 U OF CALIF GEN LIB DDCU OEPT 
1 FLHRIDA STATE U L 19 GIFTS ♦ 6XCH 
1 PSYCHOL LIB HARVARD UNIV CAM8RI0GP 

1 U OF ILL LIB SER OEPT 

2 U OF KANSAS LIB PERIODICAL OEPT 
l U OF NEBRASKA LIBS ACQ DEPT 

l OHIO STATE U Lies GIFT ♦ FXCH DIV 
l PFNNA STATE U PATTEE LIB DOCU DESK 
l PURDUE U LIBS PERIODICALS CHECKING FILE* 

L STANFORD U LIBS DOCU LIB 
L LIBN U OF TEXAS 
L SYRACUSE U LIS SER DIV 
I SERIALS RFC UNIV OF MINN MINNEAPOLIS 
L STATE U OF TDWA LIBS SER ACQ 

1 NO CAROLINA STATE COLL OH HILL LIB 

2 BOSTON U LIBS ACQ OIV 

1 U OF HIGH LIBS SER DIV 

1 BROWN U MB 

2 COLUMBIA u LIBS OOCU ACQ 

1 0!R JOINT U LIBS NASHVILLE 

2 LI8 GEO WASH UNIV MTN SPEC COLL OEPT WASH OC 
2 LIB OF CONGRESS CHh OF EXCH *■ GIFT OIV 

L U OF PGH OOCU LIBN 

L CATHOLIC U LIS EDUC C PSYCHOL UR WASH OC 
L U OF KV MARGARET I KING LIB 
1 SO ILL U ATTN LIBN SER DEPT 
L KANSAS STATE U FARRELL LIB 
L BRIGHAM YOUNG U LIB SER SECT 
L U OF LOUISVILLE LIB BELKNAP CAMPUS 
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